162

Applications in Computer Vision

Based on our assumption, for wi we formulate the ideal bimodal distribution as

P(wi|Θi) = β^k

i

2



k=1

p(wi|Θ^k

i ^),

(6.49)

where the number of distributions is set as 2 in this paper. Θ^l

k ^{= {μk}

i ^{, σk}

i ^{} denotes the}

parameters of the k-th distribution, i.e., μ^k

i ^{denotes the mean value and σk}

i ^{denotes the}

variance, respectively.

To solve the GMM with the observed data wi, i.e., the weight ensemble in the i-th

layer. We introduce the hidden variable ξ^jk

i

to formulate the maximum likelihood estimation

(MLE) of GMM as

ξ^jk

i

=



1,

w^j

i ^∈pk

i

0,

else

,

(6.50)

where ξ^jk

i

is the hidden variable that describes the affiliation of w^j

i ^{and pk}

i ^{(simplified deno-}

tation of p(wi|Θ^k

i ^{)). We then define the likelihood function P(wj}

i ^{, ξjk}

i ^|Θk

i ^{) as}

P(w^j

i ^{, ξjk}

i ^|Θk

i ^{) =}

2

!

k=1

(β^k

i ^)|pk

i ^|

mi

!

j=1

 1

Ω^f(wj

i ^{, μk}

i ^{, σk}

i ⁾

ξ^jk

i

,

(6.51)

where Ω=

2π|σ^k

i ^{|, |pk}

i ^|=mi

j=1 ^ξjk

i ^{, and mi =2}

k=1 ^|pk

i ^{|. And f(wj}

i ^{, μk}

i ^{, σk}

i ^{) is defined as}

f(w^j

i ^{, μk}

i ^{, σk}

i ^{) = exp(−1}

2σ^k

i

(w^j

i ^−μk

i ^)2).

(6.52)

Hence, for every single weight w^j

i ^{, ξjk}

i

can be computed by maximizing the likelihood as

max

ξ^jk

i ^,∀j,k

E



log P(w^j

i ^{, ξjk}

i ^|Θk

i ^)|wj

i ^{, Θk}

i



(6.53)

where E(·) represents the estimate. Therefore, the maximum likelihood estimate ^ˆξ^jk

i

is

calculated as

ˆξ^jk

i

=E(ξ^jk

i ^|wj

i ^{, Θk}

i ⁾

=P(ξ^jk

i

= 1|w^j

i ^{, Θk}

i ⁾

=

β^k

i ^p(wj

i ^|Θk

i ⁾

2

k=1 ^βk

i ^p(wj

i ^|Θk

i ⁾

.

(6.54)

After the expectation step, we perform the maximization step to compute Θ^k

i ^as

ˆμ^k

i ⁼

mi

j=1 ^ˆξjk

i ^wj

i

mi

j=1 ^ˆξjk

i

,

(6.55)

ˆσ^k

i ⁼

mi

j=1 ^ˆξjk

i ^(wj

i ^−ˆμk

i ⁾²

mi

j=1 ^ˆξjk

i

,

(6.56)

ˆα^k

i ⁼

mi

j=1 ^ˆξjk

i

mi

.

(6.57)